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This listing of claims will replace all prior versions, and listings, of claims 
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Claim 1 (previously amended): A method comprising; 

identifying, in response to a search/query, first multimedia objects having 
an associated keyword that matches a keyword in the search query and second 
multimedia objects that have contejft features similar to those of the first 
multimedia objects; 

presenting the first and second multimedia objects to a user, 

monitoring feedback from the user as to which of the first and second 
multimedia objects are relevant m the search query; and 

annotating one or more of the multimedia objects, which are deemed 
relevant by the user, with the keyword. 

Claim 2 (original)/ A method as recited in claim 1, further comprising; 

maintaining associations between the keywords and the multimedia objects, 
the associations being weighted to indicate how relevant the keywords are to the 
multimedia objects; and 

adjusting the weights of the associations based on the user's feedback. 

Claim ^/(original): A method as recited in claim 2, wherein the adjusting 
comprises ujcreasing a weight of an association between the keyword and a 
particular irfultimedia object that is deemed relevant by the user. 
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Claim 4 (original); A method as recited in claim 2, wherein the adjustii 
comprises decreasing a weight of an association between the keyword and a 
particular multimedia object that is deemed irrelevant by the user. 

Claim 5 (original); A method as recited in claim 4, farther comprising 
removing the keyword from the particular multimedia object/61 an event that the 
weight is less than a threshold value. 

Claim 6 (original); A method as recited ir/ claim l s further comprising 
training how the first and second multimedia ojagects are identified based on the 
user's feedback. 

Claim 7 (original): A method/as recited in claim 1, further comprising 
refining the search to identify additional multimedia objects that contain content 
features similar to those of the jnultimedia objects indicated by the user as being 
relevant. 

Claim 8 (original): A method as recited in claim 1, wherein the multimedia 
objects comprise ontf of digital images, video objects, and audio objects. 

Claim /9 (original): A computer readable medium having computer- 
exec^table/instructions that, when executed on a processor, perform the method as 
recited m claim 1 . 
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Claim 10 (original): A method comprising: 

iteratively retrieving multimedia objects from a database and monitoring 
feedback from a user as to whether the piultimedia objects are relevant to a 
keyword in a search query, and 

annotating the multimedia objects/based on the user's feedback, with the 
keyword. 

Claim 11 (original): A method as recited in claim 10, wherein the 
retrieving comprises using conten/-based information retrieval to retrieve the 
multimedia objects. 

Claim 12 (original); /f method as recited in claim 10, wherein the 
retrieving comprises using bothf content-based information retrieval and semantic- 
based information retrieval to retrieve the multimedia objects. 

Claim 13 (original)^ A method as recited in claim 10, wherein the 
monitoring comprises monitoring both feature-based relevance feedback and 
semantic-based relevance feedback. 

Claim 14 (original); A method as recited in claim 10 5 wherein the 
annotating is hidden from the user. 

Claim 15 (original): A method as recited in claim 10, wherein the 
annotating comprise 
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in an event that a particular multimedia object is deemed relevant by tjace 

user and is not yet annotated with the keyword, adding the keyword p the 

particular multimedia object; and 

in an event that the particular multimedia object is deemed relevant by the 

user and is already annotated with X&t keyword, strengthening an association 

between the keyword and the particular multimedia object- 
Claim 16 (original): A method as recited ja claim 10, wherein the 

annotating comprises: 

in an event that a particular multimedia dtfject is deemed irrelevant by the 

user and is already annotated with the keyword; weakening an association between 

the keyword and the particular multimedia object. 

Claim 17 (original): A computer readable medium having computer- 
executable instructions that, wheiyexecuted on a processor, perform the method as 
recited in claim 10. 

— ^ Claim 18 (original)f A method comprising: 

retrieving multimedia objects according to a content-based retrieval 
process; 

presenting/fhe multimedia objects to a user; 

monitortfig feedback from the user as to which of the multimedia objects 
are relevant^nd 

annotating one or more of the multimedia objects based on the user's 
feedback, with a keyword. 
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Claim 19 (original): A method as recited in claim 18, wherein 
monitoring comprises monitoring both feature-based relevance feedback and 
semantic-based relevance feedback 

Claim 20 (original): A method as recited in claim/1 8, wherein the 
annotating is hidden from the user. 

Claim 21 (original): A method as recite^in claim 18, wherein the 
annotating comprises: 

in an event that a particular multimedia object is deemed relevant by the 
user and not yet annotated with the keywo/fi, adding the keyword to the particular 
multimedia object; and 

in an event that the particular multimedia object is deemed relevant by the 
user and is already annotated ^ith the keyword, strengthening an association 
between the keyword and the j&rticular multimedia object 

Claim 22 (origjflal): A method as recited in claim 18, wherein the 
annotating comprises 

in an event^hat a particular multimedia object is deemed irrelevant by the 
user and is already annotated with the keyword, weakening an association between 
the keyword ^nd the particular multimedia object 

(aim 23 (original): A method as recited in claim 18, wherein the 
annotating comprises: 
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as 



in an event that a particular multimedia object is deemed irrelevant/by the 
user and is already annotated with the keyword, removing the keyworff from the 
particular multimedia object. 

Claim 24 (original); A computer readable mediuni having computer 
executable instructions that, when executed on a processor^ perform the method 
recited in claim 18* 

Claim 25 (original); A method comprising 

maintaining associations between kewords and multimedia objects, the 
associations being weighted to indicate l)Ow relevant the keywords are to the 
multimedia objects; 

retrieving a set of one or more tdultimedia objects for presentation to a user; 

monitoring feedback from tpe user as to which of the multimedia objects 
are relevant; and 

adjusting the weights of/the associations based on the user's feedback. 

Claim 26 (original); A method as recited in claim 25, wherein the 
retrieving comprises ysing content-based information retrieval to retrieve the 
multimedia objects. 

Claim 1/1 (original): A method as recited in claim 25, wherein the 
retrieving coi6prises using both content-based information retrieval and semantic- 
based information retrieval to retrieve the multimedia objects. 
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Claim 28 (original): A method as recited in claim 25, wherein/me 
monitoring comprises capturing both feature-based relevance feedba^c and 
semantic-based relevance feedback. 

Claim 29 (original); A method as recited in claim 25, wherein the adjusting 
comprises increasing the weights of the associations betweeiythe keywords and the 
multimedia objects that are deemed relevant by the user. 

Claim 30 (original): A method as recited in j^laim 25, wherein the adjusting 
comprises decreasing the weights of the associations between the keywords and 
the multimedia objects that are deemed irrelevant by the user. 

Claim 31 (original): A compter readable medium having computer- 
executable instructions that, when e^cuted on a processor, perform the method as 
recited in claim 25. 

Claim 32 (original): A system comprising: 

an information retrieval unit to retrieve multimedia objects from a database 
based on a search query; 

a relevance feedback unit to capture a user's feedback as to whether the 
multimedia objects are relevant to the search query; and 

an annotation unit to annotate, with a keyword, the multimedia objects 
based on thp user's feedback. 
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Claim 33 (original): A system as recited in claim 32, wherein the search 
query comprises a keyword-based search query having one or more keyword^ 

Claim 34 (original): A system as recited in claim 32, wherrift the search 
query comprises a content-based search query having one or more content features. 

Claim 35 (original): A system as recited in daim 32, wherein the 
information retrieval unit employs both content-based information retrieval and 
semantic-based information retrieval. / 

Claim 36 (original): A system as recited in claim 32, wherein the 
information retrieval unit comprises: / 

a query handler to handle both keyword-based queries having one or more 
search keywords and content-based queries having one or more content features of 
a multimedia object; and / 

a feature and semantic /matcher to identify at least one of (1) first 
multimedia objects having keywords that match the search keywords from a 
keyword-based query, and/2) second multimedia objects having content features 
similar to the content features of a content-based query. 

Claim 37 (original): A system as recited in claim 32, wherein the relevance 
feedback unit employs both feature-based relevance feedback and semantic-based 
relevance feedback, 

Qiaim 38 (original): A system as recited in claim 32, wherein: 
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the search query comprises a keyword-based search query having at l^ast 
one keyword; and 

in an event that a particular multimedia object is deemed relevant by the 
user and is not yet annotated with the keyword, the annotation ynit adds the 
keyword to the particular multimedia object. 
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Claim 39 (original): A system as recited in claim 32, jwherein: 
the search query comprises a keyword-based search query having at least 
one keyword; and 

in an event that a particular multimedia object is deemed relevant by the 
user and is already annotated with the keyword, the annotation unit strengthens an 
association between the keyword and the particular multimedia object. 

Claim 40 (original): A system as cfccited in claim 32, wherein: 
the search query comprises a byword-based search query having at least 
one keyword; and 

in an event that a particiri&r multimedia object is deemed irrelevant by the 
user and is already annotated with the keyword, weakening an association between 
the keyword and the particular multimedia object. 

Claim 41 (original): A system as recited in claim 32, wherein: 
the search >query comprises a keyword-based search query having at least 
one keyword; and 
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in an event that a particular multimedia object is deemed irrelevant 
user and is already annotated with the keyword, removing the keyword^fom the 
particular multimedia object. 

Claim 42 (original): An image retrieval sy^m as recited in claim 32, 
wherein the relevance feedback unit compris^a feedback analyzer to train the 
system based on the user's feedback. 

Claims 43-44 (previously withdrawn) 
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